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ABSTRACT 

Recent research in Distributed Artificial Intelligence has 
focused upon agents' interaction in Multiagent Systems. This 
paper presents a text understanding oriented multiagent 
dynamic interaction testbed (TUMIT): the theoretic 
framework based upon game theory, the free-market-like 
system architecture, and experimentation on TUMIT. Unlike 
other DAI testbeds, TUMIT views different text 
understanding(TU) methods as different "computational 
resources", and makes agents choose different TU paths and 
computational resources according to the resource 
information on the "bulletin's in their host-computer. This 
makes TUMIT like a "free market". We show theoretically 
and experimentally that if agents use multi-level of "history 
information", their behavior will tend to converge to a Nash 
equilibrium situation; and that if agents use "recall-forget" 
strategy on "history information", the convergence can be 
accelerated and the agents can acclimate themselves to 
changed environment. Compared with other DAI testbeds, 
TUMTT is more distributed, and the agents in TUMIT are 
more adaptive to the dynamic environment. 

1. INTRODUCTION 

Nowadays, the research of multiagent system (MAS) is 
becoming more and more "fashionable" in the arena of 
distributed artificial intelligence (DAI)[2). Unlike distributed 
problem solving (DPS)[1][2], the research of MAS are 
concerned with the interaction among intelligent agents|2J. 
Many researchers have borrowed the concepts of game theory 
that an agent is rational if it tries to make its payoff (or utility) 
largest [2][4][5]{7][10]f 12][13]. However, most of these 
works have three shortcomings as follows: (1) There are still 
very few testbeds, which have practical background and are 
rather general so as to verity different theoretical methods, in 
MAS as DVMT in DPS [1J. (2) Most of the theories suppose 
that the system lias a small number of agents ( 2, 3, or < 5 ), 



the agents' payoff matrix does not change, and the 
environment is statics. But in real world, the number of 
agents may be very large, and both the agents' preference and 
the environment may be dynamic. (3) Almost all of the 
theories suppose that agents have only single encounter. 
Therefore they cannot deal with the dynamic problems 
described in (2). In fact, we find that agents can learn some 
new information about other agents through multiple 
encounters, so as to adjust their choices to adapt themselves 
to the dynamical environment gradually. 

To overcome these shortcomings, we designed and 
implemented a Text Understanding oriented Multiagent 
dynamical Interaction Testbed (TUMIT). hi this paper, well 
firstly present the free-market-like architecture of TUMIT. 
Secondly, we'll show theoretically and experimentally dial if 
agents use multi-level of "history information", the agents' 
beliavior will tend to converge to a Nash equilibrium 
situation; and they can also acclimate themselves to changed 
environment. At the end of this paper, we'll compare TUMIT 
with other market-like DAI systems, e t g.. Contract Net[12], 
Enterprise[5J, etc. We show that Unlike these systems, which 
only use "bid" to make agents adaptive and self controlled, 
TUMIT uses adaptive learning strategies to make Uie system 
like a real free market that can always be equilibrium 
through "self-adjusting". 



2. THE PROBLEM OF MULTIAGENT TEXT 
UNDERSTANDING 

Text imderstanding (TU) is to transfer a paragraph of natural 
language (e.g., Chinese as in TUMIT) to a context tree, 
which can be categorized, analyzed and stored into 
knowledge base. Sometimes, multiagent text understanding, 
which means multiagent do TU concurrently , is needed, e.g., 
in news agencies and distributed libraries, that may have 
several agents in different areas to do TU tasks concurrently. 
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In TUMIT, there are 5 text understanding methods: Phrase 
Analyzer (PA), Dependency Unification Grammar Analyzer 
(DUGA), Anticipation Based Analyzer (ABA), Context 
Analyzer (CA), and Script Graph Analyzer (SGA). These 
methods form different TO paths: (ABA), (PA,CA), (PA, 
SGA), (DUGA, CA), (DUGA, SGA). Different path of TU 
corresponds to different TU quality and cost (processor, net 
work, memory, etc.). Because each TU method includes a 
large program, and a veiy large data base, it is impossible 
that every agent has all of the methods. Hie methods are only 
stored in several central computers. Thus remote procedure 
call is needed. How can the resource load be balanced in a 
system without meta-control? One strategy is to have the 
agents "call for bids", as in Contract Net [9]. However, in 
real time TU tasks, it is too costly for so many agents in a 
dynamic environment to do as in Contract Net. This paper 
presents a free-market-like system architecture to deal with 
this problem. 

3. FREE-MARKET-LIKE ARCHITECTURE OF 
TUMIT 

TUMTTs basic design idea is as follows: Hie system lias 
several public computation resource s(TU methods) stored in 
some central computers. Agents that "lodge" in different 
host-computers(Workstations, PCs) do not own any 
computation resources, and they have to "rent" computation 
resources for their TU tasks. There is no meta-control among 
the agents. They each chooses computation resources, solving 
paths and even computers independently according to its 
anticipated utility. 

TUMIT consists of 5 SUN/4 Workstations and 7 
COMPAQ/486 personal computers. Its logic arcliitecture is 
composed of 3 parts: A. Computation entities (one in each 
SUN/4 Workstation) managed by managers, which "rent out" 
computational resources (TU methods); B. Agents (10 in 
each workstation, 3 in each PC) which "rent" computational 
resources in order to do their TU tasks; C. Information 
aiuiouncement system, including a blackboard in every 
compulation entity, a bulletin in every host-computer that 
agents lodge, and a communication, mechanism 
(communication protocol, Ethernet). The logical architecture 
of TUMTT is as figure 2. 

Computation entity and Manager 

A imputation entity comprises several computation 
resources, a manager and a blackboard. When a resource is 
working, the speed of other resources that are in the same 
computer is slowed down, So in TUMIT, all of the 
computation resources in a computer form a computation 
entity managed by a manager. A manager's works are: (i) 
Simulating computational environments so as to simplify the 
experimentation, (ii) Allocating and managing computation 
resources, (iii) Sending the information about different 
computation resources (price, speed, number of agents us the 
resource, etc. ), which belong to the entity, to each agent 
group every 2.5 seconds. 



Agents and agent group 

agents that lodge in one host-computer are called an agent 
group, which has a server to receive information from 
different manager and to write to the bulletin. When an agent 
receives a TU task, it reads the bulletin, computes the utility 
of different solving paths, choose a path, and "rent" 
computation resources. . 




V Manager Q Comunication q Agent 

Protocol 

Fig I . TUMTTs Logical Architecture 



The agents thus are like consumers, and the managers are 
like sellers. Because of the scarcity of resources, one agent's 
choice will affect the speed and price of other resources, 
therefore affect the other agents' choices. Under some 
strategies, the supply and demand would tend to be 
equilibrium without meta-contral, as if there were an 
"invisible hand", which makes TUMIT just like a free market. 
We'll present our theory and experimentation of the 
strategies in such system in 4 and 5. 

4. THE THEORY OF MULTIAGENT INTERACTION 
STRATEGIES AND LARGE MULTIAGENT SYSTEM 
BEHAVIOR 

4.1. BASIC DEFINITIONS 

Model MA= ( Agt, Rsc, Tsk, u ). Agt= { a ( , a2 a„ } is a 

set of agents. Rsc is a set of computation resources, its 
element is rsc= ( r, ent ), where r is a computation resource 
and ent is the computation entity which rsc belongs to. Tsk= 
{ tj, tj, tj n } is a set of TU tasks, u is a utility function of 

an agent, u'Agt * Tsk * S-R, where s is defined as: ( Let Cj 
be the set of all of resources that agent i can select, Cj be an 
element of Cj) 

Definition 1 Situation. A situation s is an n couple (c j , 
c 2» c n ) $ is the set of all situations. 
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Assumption 1 Every agent tries to make its utility u 
maximum. 

Definition 2 Pure Equilibrium Situation ( PES ). A 
situation s* is a PES if for every agent i 6 Agt and tj 6 Cj, 

u( i, tsk, s*||tj) < u( i, tsk, s* ). 

Sometimes PES may not exist. As in game theory, we define 
mixed choice, mixed equilibrium situation, and mixed 
equilibrium situation. 

Definition 3 Mixed Choice and Mixed Choice Situation. 
For every agent i € Agt, xj an i's mixed choice if Xj is 

probability distribution on Cj: xp (xj j, xq x un ), x^ > 0, 

k=l m, £ Xjj.=l . i chooses cy, under the probability x^. 
x= ( Xj, xj, ... , x n ) is a mixed choice situation. 

Denote: x\\ifi x v .... x { . v x ; , x i+1 , .... x Q ). This means 
that in a mixed choice situation x, agent i changes its choice 
from xj to . while other agents do not change their choices. 

m l m n 

U (i,uk..x)= £ £ «(i.ttk.. b k k )x k x k .(Sfci-kn means 

k,=l k =1 1 2 n I n 

i chooses the kjth element in Cj as its pure choice. ) 

Definition 4 Mixed Equilibrium Situation or Nash 
Equilibrium Situation (NES). A mixed situation x* is a 
NES if for every agent i t Agt and z x C Xj, u( i, tsk, x*||zj ) 
<u(i,tsk, x* ). 

Lemma 1 In every M, there exists at least an NES. ( A 
simple corollary of Nash Theorem [10] ) 

4.2. UTILITY-QUALITY-TIME FUNCTION 

Most MAS researchers use payoff or utility matrix (discrete 
points) to describe the agents' different preferences. However, 
very few of them show how an agent can get these matrices 
automatically, and how to deal with "new interaction points" 
if the environment is changed. This paper presents a utility- 
quality-time Amotion that is continuous so as to compute 
utility of every point. We show that an agent's utility u is its 
earnings E minus its total cost (in TUMTT, total cost is the 
sum of cost of processor: p, memory: m, and net work: np). 
The functions. are shown as in (I )-(5). 

u = E- p-m-np 



(i) 

(2) 



p = lim X PP(t, ,n % a)&t = £pp(t,n 9 a)dt 



m = j''MP(t,n,um)dt 
np = f' NP(t,n\,n2,un)dt 



(4) 
(5) 



(3) 



B, kj, k£> Tfc and t^, are coefficients corresponding to TO 

tasks. B is a basic reward to an agent if it finish a TU task at 
time tfc and at quality r^. If the agent finish the task before t^ 

((t-tjj)O) or if it finish the task at a quility higher than r^, it 

can earn some extra reward tliat is little. However, if the 
agent finish the task later than t^ or if the quality is lower 

than rjj, the reward it earns will be reduced very much, thus 
it may earn very little or even cannot* earn any reward. 
Therefore an agent has to choose computation resources so as 
to finish the TU task at a high quality in time and minimize 
the cost. 

More detail of utility-quality-time function is in [7]. 

4.X CONVERGE TO EQUILIBRIUM SITUATION: 
USING HISTORY INFORMATION AND RECALL- 
FORGET ALGORITHM 

hi large MAS systems without meta-control, e.g. TUMTT, it 
is too costly for all agents to make joint decisions through 
negotiation. Every agent has to choose solving paths 
according to its knowledge and anticipation about the 
resources in the system. Because in TUMTT, the parameters 
of the resources, speed, price, etc., are decided only by the 
agents' choices, an agent has to predict the other agents' 
choices before it makes its decision. This paper presents 4 
strategies for agents to make decisions. Well firstly present 
the concept of NES in TUMTT. 

Assumption 2 hi TUMTT, two agents have the same utility 
if they are in the same situatiou. 

Definition 5 Situation Equivalence (SE) . If for every 
agent, it can get the same utility at two different situations, 
the two NES are called SE. 

Definition 6 Situation Equivalence Class (SEC). A SEC is 
a set of all situations that are SE. 

Lemma 2 Ail agent lias the same choice at the situations in 
the same SEC. ( A corollary of Definition 5,6) 

Strategy 1 Random Decision 

Every agent chooses solving paths at the same probability. 

Using this strategy, agents cannot get to NES. Moreover, 
because every resource is used at the same probability, the 
resource that is slower than others ( because of the computer 
and the method itself ) may be overloaded. 
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Strategy 2 Decision According To The Newest Information 
In Bulletin. 

An agent believes that other agents have "pure" choices 
which is the same as their last choices. However, TUMIT 
cannot change the bulletins as soon as the parameters of the 
resources change. Thus there exists an information delay, t , 
in every host computers. 

Theorem 1 Under strategy 2, if t is small enough, the 
choices of all of the agents in TUMTT tend to converge to an 
NES. If t is large enough, the system cannot converge and 
will oscillate. (Proof in [8]) 

We find that if the agents can get to an NES, the speed of the 
convergence is higher than that under strategy 3 and 4. But 
strategy 2 cannot assure convergence. In many real systems, 
t is very large because of the delay of network and other 
factors. Thus the system may oscillate as described in the 
"computational ecology"[5|. 

Strategy 3 Decision According To All History Information 
Every agent anticipates others' choices according to their " *» 
history mixed choices", x, (x;j > 0, Z Xjj=l), defiued as 
follows, 

Definition 7 <» History Mixed Choices ( ~ HMC) 
Suppose in (k-1 )th encounter, agent i e Agt chooses cjj £ C v 

iu kth encounter, i's ~ HMC % t is: 1° if k=0, Xjj°=I/u ; 

2° if k=l, Xjj 1 ^ G*h)> Xih 1=1 '. 3° if k>l. 

Xij k =Xij k_1 (k-l Vk (j * »)> x^Kx^kk-l HI yk. 

Theorem 2 Strategy 3 can always make convergence to an 
NES. 

Proof (abbreviated because of the limited paper) : Let x* be 
the NES uearest to x 1 flx*-* 1 ) to be rninimiun ), there exists 
an integer M, when k > M, xy k X), if and only if Xjj*>0. [8] 
There exists an integer N, for every k, there exists an integer, 
n < N, x k+ti =x-[8). Let x kl =x\ x k,+n =x*, (, uu < N ) 
dj=MAX() < nl < n (|x k l + «l. x1X d 2 =MAX 0 < m , < 

m (|x k,+n4inI -x*|). According to Definition 7, d2/dl < 

(k 1 +l)/(k 1 +n+m). (101 Therefore when k- ~, |x k -x"|-0. 
EOF 

x in strategy 3 cannot be used directly in TUMTT. Instead, we 
have an algorithm (omitted because the limited paper) to 
make agents compute the SEC of x k according to bulletins. 
Lemma 2 and Lemma 3 assures tlic equivalence of the 
algorithm and strategy 3. Strategy 3 can always make 
convergence to an NES no matter how large r is, because 



every agent considers all other agents' history choices. But 
the speed of convergence is very slow as shown in the proof 
and in 5. Moreover, when k is very large, if the environment 
changes, the agent cannot adapt to the new environment 
because the agent considers too much about the old history 
information. 

Strategy 4 Recall-Forget Strategy 

The definition of x in strategy 3 is changed to x as follows: 
1° if k=0, 1: x k =x k ; 2° if k>l, x_ij k = 6 Xjj k " ' (k- 1 )/k ( j 

*h ); XiliN^x^'+k-SCk-Oyk, where 8 is called 
-recall coefficient M ( 0< & < 1 ). 

When 5 = 0, strategy 4 is the same as strategy 2; to the other 
extreme, when 6-1, strategy 4 is the same as strategy 2. 
When 0<8 <1, old history information is tending towards 

being "forgotten" (because if k — 00 , 6 k ~* -* 0 ). Thus 
strategy 4 can accelerate the convergence to the NES and 
make the agents acclimate the changed environment. The 
smaller & is, the faster convergence is made. But too small 
6 cannot assure convergence. Therefore, strategy 4 is a 
compromise of strategy 2 and strategy 3. As in strategy 3, we 
also have an algorithm to compute the SEC of x k - 

Theorem 3 In a MA defined as in 4, 1 , there exists a 6 q, 0 

< 6 q < 1 , for every 6( 6 q < 5 < 1). strategy 4 which 

uses 6 HMC can make agents converge to an NES. (I*roof 
omitted because of the limited paper ) 

5. EXPERIMENTATION ON TUMIT 

We have done many experiments on TUMIT, Some are 
presented, in this paper. In order to simplify the experiments, 
5 computation resources are in 5 SUN/4 workstations 
respectively. (Thus they do not affect one another ). 20 agent, 
each has 64 TU tasks, work concurrently. Every 5 seconds, 
each agent receives a TU task. Information delay time, t . is 
controlled by the managers by delaying the bulletins. 

(1) Strategy 1. (Analyzed in 4.3 ) 

(2) Strategy 2. 

When t = 2 seconds, the agents' choices can converge very 
quickly. When t= 15 seconds, the agents' choices oscillate 
as in figure 2. 

(3) Strategy 3. 

The agents' choices can converge slowly as in figure 3. 
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0 3 6 9 12 21 24 

Fig 2. The curve of SGA in strategy 2 

x 's axis is 1(105 J, /s axis is speed(level). "level" is defined in 1 1 1 J. Originally, 
there are 5 curves that describes 5 resources. However, we find thai it is clearer 
for the readers to have only one curve in this figure. The curve that is in Fig 2, and 
Fig 3 . Fig 4 15 that of SGA in computer 4. The curves of other resources is similar 
to this one. 

(4) Strategy 4. 

When S = 0.8, the chart is as figure 4. But when o =0.3, the 
convergence is made very quickly. When 6 =0.05, t =5 
seconds, the figure is very similar to figure 2 that shows 
oscillation. . 




Fig 3. The curve of SGA in strategy 3 

For x, y axis and other information . sec Fig 2 




0 3 6 9 12 15 

Fig 4. The curve of SGA in strategy 4 

For x y axis and other information, see Figure 2 



6. CONCLUSION 

We liave mainly concentrated on two points which are never 
seen in other researches: the free-market-like TUMTT 
architecture and the use of history information in multiple 
encounters in MAS. Compared with the classic DPS systems, 
TUMTT does not need central control, thus is more adaptive 
to the real world. There are some other market-like systems, 
e.g.. Contract Net [91, Enterprise [5], etc. They all use the 
process "calling for bid", o kind of negotiation, to make 
agents adaptive and self controlled. However, as we show in 
this paper that in many times, a large MAS system cannot -get 
coordination through negotiation. TUMTT uses adaptive 
learning strategies to make the system like a real free market 
that can always be equilibrium through "self-adjusting". 
Furthermore, TUMTT has a comprehensive background, TU, 
which makes it very comprehensive. More experiments, 
especially those about the value of 6 are to be done in. our 
later research. 
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Abstract 

In recent years, electric utility industry in the whole 
world has been facing pressure to be deregulated or 
restructured in order to increase its efficiency, to 
reduce operational costs or to give consumers more 
alternatives. The once centralized system planning 
and operation management must be remodelled to 
adapt to the new market environment. Subject to the 
unavoidable constraints, such as, capacity of 
generation stations, physical limitations of the 
transmission lines, and demand on days-ahead 
scheduling, the current trading mechanism needs to 
be revised so that any party can involve in this free- 
market decor. This paper presents a multi-agent 
approach to resolve the multilateral trading 
problem. We have implemented a prototype based 
on Bilateral Shapley Value and Internet 
technologies. The prototype has been tested with a 
classical six-bus system. 

1. Introduction 

Deregulation and restructuring of utility 
industry has been an important area for the past 
decade. Regulatory body, operation environment, 
technologies, definition of boundary, and the nature 
of electricity business have changed very fast. 
Therefore, it is very difficult and almost impossible 
to use the old rules or regulations, which were 
designed decades ago even before the creation of 
computers, to manage and control the current power 
industry [1, 34]. In the past, production and 
distribution of electricity had been considered as 
natural monopoly due to the economic scale [33]. 
New power plants and transmission systems could 
cost several billion dollars. Such franchised 
monopoly, under protection and insulated from 
competition, obviously had a much weaker incentive 



in controlling costs and avoiding poor decisions as 
those competed in open markets. Therefore, 
customers have been imposed excessive costs which 
were unfair and unjustifiable [6], For example, poor 
decisions in 1970s in expanding power generation 
capacity by building more nuclear power plants had 
forced the consumers to pay higher prices to recover 
the additional costs, such as, the case in Washington 
state. One of the major motivations to deregulate 
and restructure the power industry is to avoid the 
same mistake [36]. 

As innovations in electrical engineering, 
material sciences, and supporting information 
technologies have been developed rapidly, the low- 
cost production and transmission of electricity have 
become possible [6], Therefore, several countries, 
such as, United Kingdom, New Zealand, Norway, 
Chile, and Argentina, have taken significant steps 
toward the deregulation and privatization of their 
electric power industry [16, 29, 33]. 

Generation, transmission, and distribution 
services used to be bundled together as a single 
commodity. After restructuring or deregulation, 
they could be treated as individual service providers 
and priced their services separately. Therefore, 
more participants, many of them have been 
previously excluded in making decision on behalf of 
themselves, could participate in the trading to create 
a more cost-effective and more efficient market [7, 
27]. Expansion of the power system has been driven 
by the market demand. Security, reliability, and 
economic issues have to be separated so that the 
conflict of interests in the power dispatching can be 
avoided. This was the major incentive to the 
development of the Independent System Operator 
(ISO). How to carry out a smooth transition from 
the regulated and protected environment to a more 
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open and competitive market, ISO may have an 
important role to play [1, 17]. 

In a restructured market, information structure 
and decision making process have become more 
decentralized and more distributed. Therefore, the 
requirements on the information processing and 
communication among the participants have also 
been changed to reflect the changes the market 
structure. Decentralization of planning, 

management and operation have created a demand 
for new methodologies, models and technologies. 
For example, a workshop had been organized by the 
joint efforts of Pserc (The Power System 
Engineering Research Center) and EPRI (The 
Electric Power Research Institute) to identify the 
tools and systems in engineering and economics that 
needed in the future [34]. 

In such decentralized market, each participant 
should allow to stay autonomously in a free and 
open market. In addition, they may be 
geographically dispersed, Internet and agent 
technologies can support their communication, 
negotiation and coordination [3, 20, 24, 35, 37]. 
Internet and agent technologies can be used together 
to develop systems to support both planning and 
operation of utility market [15]. However, the 
requirements on supporting operation are much 
higher, because a decision has to be made within 
seconds if there is a demand for balancing the 
loading. We will solve the operation problem in our 
future research. In this paper, we focus on system 
planning phase. 

To prove that Internet and multi-agent systems 
may be a new alternative to the planning for system 
expansion, we developed an Internet-based multi- 
agent system to help participants identify better or 
more appropriate partnerships (or coalitions) within 
the electricity market, such as, suppliers and 
consumers. The core of the multi-agent system is a 
flexible multilateral trading scheme, which was 
developed based on the Bilateral Shapley Value 
(BSV)[30, 38]. 

Market structuring of power industry and a 
literature review of the deregulation and 
restructuring of power industry will be discussed in 
Section 2. The rules for the will be provided in 
section 3. The agents for the multilateral trading 
will be discussed in section 3. 

2. Agents, Cooperative Game Theory 
and Multilateral Trade, and ISO 

In this section, we will briefly discuss the 
research in software agents, cooperative game 
theory, multilateral trading, independent service 
operator (ISO) and schedule coordinator (SC). 

2.1 Software Agents 



Many applications have been developed 
recently to remove the burdens of human beings 
from those tedious and repeating tasks. One of such 
applications is the software agents. If agents have 
special capabilities, such as, learning, proactive 
decision making, and communication, they can be 
called intelligent agents. Marvin Minsky said [25]: 

"My on-line Random House dictionary includes 
many senses for agent: 

When you use the word "agent " in any of these 
ways, there is no implication that what the agent 
does is simple. They all suggest the agent is seen as 
having some specialized purpose. If you need help 
with making investments, you call a financial agent. 
If you *re looking for a job, you call an employment 
agent. In the present-day jargon of computer 
interface agents, the word is used for a system that 
can serve as a go-between, because of possessing 
some specialized skills. " 

The agent approach to support intellectual 
work, collaboration, information searching and data 
analysis has created many successful applications 
and gained significant attention over the past few 
years. Aparicio has identified eight major 
application areas that relate to agents [2], For 
example, manage systems and networks, manage 
workflow, and support electronic commerce. 

These applications represent a fundamental shift 
in the human-computer interaction paradigm. As 
stated in a recent Science article: "...the agent 
approach transforms the computer user from a 
worker into a manager [35]. You delegate tasks to a 
set of agents who do things for you." One of the 
basic ideas of the agent research is to develop 
systems which "engage" and "help" different end 
users to reach their goals [28]. 

An "agent" can be a program that operates 
autonomously to accomplish unique tasks without 
direct human supervision (similar to human 
counterparts such as real estate agents, travel agents, 
etc.) [23]. For example, a "spider" who looks for 
relevant information in the cyberspacefl 1], an 
scheduling agent schedules meetings based on 
constraints or patterns that the agent had learned 
previously, or an information or e-mail filtering 
agent who filters news group articles based on the 
"induced" (or learned) users 1 profiles [24]. 

Agents receive assignments and instructions 
from the owners, then based on the models or 
knowledge bases, accomplish the assignments. 
Some agent researchers attempt to address the 
question: "how should agents interact with each 
other to form agent teams on the Internet?" Other 
researchers are more concerned with designing 
agents who are "robust", "intelligent", and 
"human-like" [28]. 
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Multi-agent systems are special type of agents, 
which more focus on coordination and 
communication among agents either to 
collaboratively accomplish a task [19, 20, 21, 31]. 
In our multi-agent system, the agents are naturally 
the generators, customers and coordinators (to be 
explained later). The objective is thus to derive a 
workable and profitable coalition partnership under 
the fair play practice subject to power generation 
and transmission conditions. 

Communication and cooperation are two most 
important capabilities to the multi-agent systems. 
Multi-agent systems are designed to have the 
capability to either, collaborate, for example, 
decompose a problem and jointly solve the problem, 
or compete, such as, search for the best deals for the 
users. The term cooperation used in this paper is 
assumed to include both collaboration and 
competition. Communication is vitally important by 
which relevant information to support cooperation is 
exchanged. KQML (Knowledge Query and 
Manipulation Language) is a language that supports 
the communication among agents [12]. However, 
agents must do more than just communication. 
Rational agents must be able to cooperate and 
negotiate with each other. Design of the 
communication and negotiation protocols is 
important. However, so far there is no protocol that 
dominates this field. One language developed by 
Barbuceanu and Fox [4] called COOL, which is an 
extension of KQML. COOL allows the agents to be 
developed with the capability to make proposals and 
counter-proposals, accept and reject goals, notify the 
other agents of goal cancellation or creation. 

2.2 Game Theory and the Multi-Lateral 
Trading 

Game theory has been an important research 
area to both mathematics and economics for the past 
four decades. They can be applied to solve most 
decision making problems by creating games where 
the following three conditions are satisfied: 

1. There are two or more autonomous 
decision makers, called players; 

2. Each player has a choice of two or more 
ways of acting, called strategies, such that the 
outcome of the interaction depends on the strategy 
choices of all the players; 

3. The players have well-defined preferences 
among the possible outcomes, so that numerical 
payoffs reflecting these preferences can be assigned 
to all players for all outcomes. 

Application of game theories to solve 
distributed artificial intelligence (DAI) problems, or 
more precisely, the coordination and collaboration 
of agents, is a new but promising research area. We 



therefore use the multi-agent approach and game 
theory to model the core of the utility market - the 
multi-lateral trading mechanism. 

Multi-lateral trading mechanism has played an 
important role in planning and operation of 
electricity industry. Game theories, especially the 
cooperative game theories, have been used 
extensively to identify the trading partners in various 
trading markets [13, 38]. The purpose of the 
multilateral trading game is to search for the best 
coalitions for each player in the game. 

However, just searching for the best coalition is 
not sufficient. How to fairly allocate the costs and 
the payoffs among the players can be even more 
important. Among various models, Shapley Value 
and Bilateral Shapley Value have shown great 
potential and been used by many researchers [38]. 

Forming coalition and identify business partners 
is important in the deregulation and restructuring of 
power industry [3, 18, 32]. The reason is that the 
old boundary or regulations have been removed and 
participants need to find new partners to protect 
their long-term interests, for example, stable ,supply 
of low-price electricity. Many approaches have 
been used to determine the coalitions or planning for 
the network expansion, for example, sensitivity 
analysis [5], Branch-And-Bound Integer 
Programming [22], Discrete Dynamic Optimization 
[10], and Least-effort Criterion [26]. However, most 
of them are centralized approaches and they do not 
support the decentralized nature in the utility 
markets after their restructuring or deregulation. In 
the following, we will briefly discuss the process of 
forming coalition. 

There are four phases of cooperation and 
coordination among agents in the multi-lateral 
trading system to form coalition [20]. To simplify 
the development of the prototype, only bilateral 
negotiation is considered in which exactly two 
participants are involved in each discussion or 
negotiation in forming coalitions. 

1. Information Derivation: Each agent submits its 
local trading requirement such as costs of power 
generation or profits of selling electricity to an 
independent service operator (ISO) agent who 
ensures the feasibility and fairness of trades. It then 
receives the global information, such as, the average 
cost of using transmission network. 

2. Communication: Every agent exchanges its 
information with other agents directly or indirectly 
through a coordinator agent. Each agent 
continuously gathers information from the ISO 
concerning the market environment, for example, the 
network reliability. 
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3. Computation: Every agent calculates the total 
profit and its own share if teaming with other agents. 
A preference list of other agents will be created in 
the order of most positive total profits. 

4. Bilateral Negotiation: According to the 
preference list, each agent extends offer to others on 
the list. At the same time, it may accept another offer 
which will improves its position and consequently it 
has to decline the current offer. The process will be 
iterated until a pre-set duration has arrived and after 
that, no further negotiation is allowed. 

After these four steps, the system would come 
out with the coalition and the list of players in the 
coalition. The next phase is the costs or payoffs 
allocation. 

Costs allocation is an important step after 
formation of coalition. Since each player has 
contribution to the total payoff of coalition, 
therefore, it is necessary to go through a process to 
allocate the costs or profits according to individuals' 
contributions. During the computation step (Step 3), 
each agent has to know its own share of profit. There 
are many solutions to derive the payoff vectors, for 
example, the core, stable set, and Shapley value etc.. 

In order to be consistent with how the coalition 
was formed, we again use the Bilateral Shapley 
Value (BSV) model [20] to identify the relative 
weighting of their contributions. For example, 
consider a network of four agents namely A, B, C 
and D. The coalition sets are: {{A},{B},{C},{D}} 
-> {{A,B},{C,D}} {A,B,C,D}. The profit for 
{A,B} in {A,B,C,D} will be equal to its BSV, say 
Pjw and the same applies to {C,D}. Then the share 
of A in {A,B} is given by: 

BSV of A in {A,B} 

BSV of A in {A,B} + BSV of B infA.B}^ 

The profits for all the agents are calculated 
similarly. Besides Bilateral Shapley Value, the 
concept of Kernel in cooperative game theory can 
also be used to allocate profit or costs [32]. 

2.3 Independent Service Operator and 
Schedule Coordinator 

In a multilateral trading, the major participants 
are generators, customers, schedule coordinators and 
the independent system operator (ISO) [17]. The 
ISO administers the security and reliability of power 
generation and transmission, ensures that all 
constraints and limitations are met. In addition, it 
releases information to all the market participants, 
which include hazardous conditions, such as, 
transmission failure and over-generation. 

A schedule coordinator (SC), or simply a 
coordinator, is responsible for the formation and 



coordination of a coalition of generators and 
customers. It promotes multilateral trading which 
involve two or more players to guarantee that the 
sum of power generation equals the sum of power 
consumption plus the losses due to transmission. An 
SC also has another goals, such as, search for 
profitable trades and insure that the trades are 
feasible based on the available information and 
constraints. 

In fact, a SC may as well be a generator, a 
customer, a broker, or a power exchange (PX) [17], 
which serves as a daily spot market to balance the 
supply and demand of electricity with publicly 
posted prices. A sample dialogue in Figure 1 
illustrates a mode of information flow between a 
coordinator and a participant in which a participant 
submits a request and the coordinator, after 
consulting the ISO, replies with an endorsement or a 
counter proposal if the request cannot be granted. 

The ISO, SC, generators, customers, and 
potential market participants all require intensive 
communications in order to establish schedules, 
handle transactions, and perform analysis during the 
trading process. The trading must be based on the 
equal opportunity and fair competition principles. 
As such, each participant should be autonomous and 
be responsible for own decision making. In this 
respect, intelligent agents or multi-agent systems are 
found to be suitable to model such complex, 
independent and yet cooperative behavior. 



schedule 
coordinator 


^ Connect - 


potential 
participant 




— — -____Ac«£(ed 


open 

connection 












negotiation 
& 

bargaining 




Ctose____ 


close 




4 " 


connection 



Figure 1. Coordinator dialogue sample 

3. An Example 

In the above section, we have introduced the 
model and process to form coalitions. In order to 
illustrate how they work, we use an example in this 
section. Throughout this section, a simple four-bus 
system is used to discuss how to model the multi- 
lateral trading system. An agent in the game can be 
a power station (generation) or a group of consumers 
(consumption). For simplicity, we exclude the cases 
of fractional power generations or consumption. We 
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also assume that any set of power generators or 
consumption loads attached to the same bus will be 
represented by the same agent. 

In the following, we will examine how the 
coalition is formed with a simple 4-bus network. 
Figure 2 depicts a network with bus nodes © and CD 
represent the generators while nodes ® and © the 
customers. Other information required is the 
reactance/p.u. for each transmission line and a 
transfer admittance matrix (not shown) used in 
calculating power flow. 
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Figure 2. A 4-bus system example 

To compute the total profits of a coalition, we 
also need to consider the transmission cost for every 
bus line. Here we assume that the cost of 
transmission varies linearly (Figure 3) with its line 
flow and that the transmission network has infinite 
capacity. ^ 

cost of using 
the 

transmission 
line 




Figure 3. Linear transmission line cost 
model 

Figure 4 displays the global and local 
information that define the 4-bus system where m in 
the global table denotes the slope of the above 



linearity relationship. For the local information, the 
costs of generation for both generators are the same 
(i.e. 0.05), as well as the benefits of consumption for 
the customers (i.e. -0.06). 
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Figure 4. Global and Local information 

We formulate the problem as an optimization 
problem to find the power injection by the entities in 
a coalition C, p c resulting in optimal profit. The 
mathematical form of the objective function is given 
by 

" C{p c ) + l T -\H-p\ 

J 



Minimize 

subject to 
C = 



W • p c — 0 where 

vector of cost / benefit 
functions for agents in C 
pc - vector of power injections by 

agents in C 
/ = vector of all transmission 
line costs 

H = transfer admittance matrix of the 

transmission network 
p = vector of all net power injections 

at all buses by agents in C 
u = column vector of all ones 

Profit of coalition, therefore is calculated by the 
following 
equation, 

v(C) = -u T C(p C(min) )-l T ]H- P(min) \ 

Based on the above data and the optimization 
criteria, the outcome of coalition can be calculated 
and shown in Figure 5. The difference in the profits 
for {1,3} and {1,4} is due to the higher transmission 
cost induced for {1,3} where power needs to flow 
through both lines 1 and 3. Also notice that the 
grand coalition {1,2,3,4} naturally results in the 
global optimum as this is precisely the monopolistic 
case before deregulation. 
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Figure 5. Coalition Formation of the 4-bus 
network 

4. Implementation 

In the previous section, we have used a four-bus 
example to illustrate how to form best coalition and 
how to allocate profits or costs among the players in 
the same coalition. In this section, we will discuss 
how to develop the multi-lateral trading mechanism 
based on the multi-agent approach to solve the same 
problems. 

There are several systems been developed to 
help the decision makers in their planning of the 
expansion or operation of power system, for 
example, systems that based on Ptolemy or other 
graphical software [8, 9]. However, they are 
centralized approaches as mentioned earlier. As 
mentioned earlier, the power system planning and 
operation will be decentralized across under the new 
regulations or new market structures which are 
geographically dispersed. To meet such challenges, 
such as, coordination and dissemination of 
information, we should develop new systems and 
new methodologies. In the following, the 



requirements of coordinated multilateral trading, the 
protocols of communication, the structure of 
information, and the support for computation will be 
discussed. These requirements were the foundation 
for the selection of technologies to develop the 
target system. 

1. Coordinated Multi-lateral Trading 

The multi-lateral trading model was developed 
based on two important entities: the Independent 
System Operator (ISO) and the Schedule 
Coordinator (SC). ISO controls the transmission 
dispatching to ensure the security and reliability of 
power transmission. Only when security is 
threatened, ISO would intervene to reduce or adjust 
the schedules submitted by the SC. SC is the one 
who, based on economic analysis, proposes 
coalitions of generators and consumers. It also 
arranges multi-lateral tradings to make sure that the 
sum of power generation equals or more than the 
sum of consumption plus the losses due to 
transmission. 

2. Protocols and Infrastructure of 
Communication 

During trading periods or planning stage, ISO, 
SC, generators and consumers, and other potential 
participants require extensive communication to set 
up schedule, handle transactions and conduct 
analysis. The protocols and infrastructure at this 
moment still wide open for research. In order to 
provide high-quality and stable communication and 
to use the existing communication infrastructure, we 
selected applications, such as the World-wide Web 
(WWW), and protocols, TCP/IP, that run over the 
current Internet. 

3. Structure of Information 

On the electricity market, ISO has to release 
information about loading condition of power 
transmission to SC. Based on the information, SC 
arranges trades and forward the messages back to 
ISO. In such case, ISO is a server who receives the 
requests from the SC and other potential market 
participants responds accordingly through the 
protocol as discussed above. The SC is also a server 
who receives requests from the potential trading 
partners to monitor the request queues. Therefore, 
the whole system is best to be modelled with client- 
server architecture. To support the back-end 
database management, we selected the mSQL to 
provide fast access to the stored data. 

4. Support Computation 

ISO ensures security and reliability of power 
transmission, maintains real-time load balance, and 
guarantees that constraints are met. All these 
operations are computation intensive. In order to 
reduce the turnaround time of computation, we had 



» 
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to design our system so that it supports distributed 
processing. In the real-world cases, an ISO 
distributes processing loads to several processors, 
which can be heterogeneous and geographically 
dispersed. There are many approaches and 
applications to solve the computation problem which 
encountered in such distributed environment. In 
order to save the time and efforts in the development 
of our system, we selected existing software tools to 
perform the computations, such as, optimization. 

The multi-agent based multi-lateral trading 
mechanism is designed to allow decision makers to 
explore more alternatives in planning for new 
expansion or searching for new partners. Based on 
the above requirements, it was developed with the 
following technologies: 

1. Java 

Java is an object-oriented and distributed 
programming language. It was specifically designed 
to develop systems that run on the Internet. Its 
network centricity, sandbox security model, and 
platform independence make it an ideal candidate 
for the development of agent-based tools. In 
particular, the object-oriented (OO) nature of Java 
enables the modelling of generic agents. Any 
specific agent can be modelled based on the 
inheritance characteristic as existed in class. 

2. JATLite 

Our multi-agent system is developed based on 
JATLite which provides a set of lightweight Java 
packages for the development of multi-agent 
systems. JATLite, developed at the Stanford 
University, supports several layers of agent 
communication through message passing. 
Knowledge Query and Manipulation Language 
(KQML) is one of these layers that we have been 
used in the system. 

3. Ptolemy, Tcl/Tk, MATLAB®, and mSQL 

While the multi-agent model being more or less 
a front-end application, the entire system indeed 
includes a host of integrated tool set for back-end 
support and processing. Ptolemy offers a flexible 
framework supporting heterogeneous system 
specification, simulation, and design. Each model of 
computation is a domain, which consists of an 
extensible library of functional blocks. We have 
employed a distributed optimization tool built with 
Ptolemy to solve the transmission constrained 
economic dispatch problem in power system 
analysis. 

MATLAB has been used to compute the 
optimal profit for a participant in forming coalition 
which can be formulated as a linear programming 
problem. mSQL is a distributed lightweight 



database engine providing fast access to relational 
data organization. All power line and network 
characteristics are captured in the database. Finally, 
Tcl/Tk supplies the glue logic to link the above 
modules dynamically and can even be integrated 
with the Java/Internet environment. 

As part of the system, we have also 
implemented an prototype of the ISO, from which 
some of the less sensitive information can be 
disseminated to all the participants through the 
Internet. 
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Figure 6. ISO web page 



Figure 6 shows a simple ISO web page which 
allows any participant to query a particular power 
line network, electrical specification of generators 
and loads , and the loading vector of transmission 
lines. 

Figures 7 and 8 tabulate the detail transmission 
line topology and power characteristics of a 6-bus 
sample taken from [34]. All the data are resided in 
the ISO database. 
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Figure 7. bus information web page 
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Figure 8. Network information web page 

For simplicity there are only two SCs 
representing all the six generators and customers. 
The SCs establish trade with the potential partners 
through the simple dialogue as shown in Figure 1. 
After bargaining and negotiation with the potential 
partners, the SCs developed preferred schedules 
resided in the databases of the two SCs. 

The ISO collects schedules from both SCs 
during each time period and load them into the 
database. Before the next operating day, the ISO 
sums the generation and consumption on each bus 
for each time period. All the transfer of information 
are done through the Internet and among different 
agents. The ISO then check the feasibility of all the 
schedules. To check if trades are feasible, the ISO 
calculates and solve the network of flow problem 
based on the information in the database. If there is 
a overloading in transmission, the ISO initiates the 
congestion management protocols to relieve the 
congestion. In such case, the ISO has to reduce the 
generation and consumption as requested by the 
participants. Such process can be a repeated process 
until overloading is no longer exists. 

Such process is running in the backend, 
therefore, it is necessary to create an user interface 
for the users to monitor the progress and the 
activities among the agents. The interface was 
developed with Java and JATLite so that it can be 
accessed through Internet. The agent server (Figure 
9) is responsible for disseminating global 
information from database, bookkeeping the 
registered customers, obtaining coalition status from 
other agents, and terminating the negotiation cycle. 
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Figure 9. A server agents panel 

Figure 10 shows the process of one agent 
requested to join the market. On figure 10, it is easy 
to see that the message was sent from the agent to 
the ISO. ISO received the request, processed it and 
sent the acceptance message back to the agent. 
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Figure 10. A customer agent's panel 

Figures 11 and 12 shows the panels of bus 
nodes CD and ® respectively after negotiation among 
the agents. From both panels, users can monitor or 
trace the preference lists, the profits that calculated 
based on the information the agent received from 
other agents, and the decision or message that sent to 
the other agents. 
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1 ©4.064 
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b: [0.06. 0.05] 
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113.636 
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Fig. 11 Bus Node @ after negotiation 
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preference Use [gen2] 
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busType: O, 3] 
a: U 20, 120] 
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Fig. 12 bus node ® after negotiation 



5. Conclusion 



In this paper, we have presented a multi-agent 
based approach to assist the planning and operation 
of utility market. Such approach is fully centralized 
and that is closer to the markets after deregulation 
or restructuring. This approach can be used not only 
to support the operation, but also as a simulation 
system for the users to explore different alternatives 
in their planning stage. 

The major benefits of using such approach are 
the following: 

1 . Decision makers can participate in 
modelling the behavior of the agents, for example, 
costs, goals, and preferences, and let the system 
searches the best partners. Compare with a 
centralized approach, each agent has higher freedom 
in designing its own strategies or negotiation rules. 
Normally the centralized approach just assume that 
all the agents or agents who belong to the same 
category, such as, generators, use the same set of 
rules. 

2. It allows the users to explore different 
alternatives within short time. For example, if one 
agent has captured the strategies of the other agents, 
then the agent can generate different scenarios to 
identify the best strategy for bargaining or 
negotiation. 

The implementation has considered Internet or 
Intranet connectivity by using Java as the core 
development tool. Therefore, our system can over 
the barriers that come from distance and also the 
users do not need to invest on acquiring dedicated 
communication lines. 
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